2 MECHANISMS OF MULTITASK BACKPROPWe

Hinton 6] proposed that generalization in artiicial neural nets should improve if nets learn to represent the domain's underlying regularities. Abu-Mustafa's hints work 1] shows that the outputs of a backprop net can be used as inputs through which domain-speciic information can be given to the net. We extend these ideas by showing that a backprop net learning many related tasks at the same time can use these tasks as inductive bias for each other and thus learn better. We identify ve mechanisms by which multitask backprop improves generalization and give empirical evidence that multitask backprop generalizes better in real domains.